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DISTRIBUTED PROCESSING OF HIGH In order to facilitate such communication, industry and 

LEVEL PROTOCOLS, IN A NETWORK international standards bodies have established sets of func- 

ACCESS SERVER tional requirements, conventions or rules that govern the 

transmission of data over both telephone and packet 

CROSS-REFERENCE TO RELATED s switched computer networks. These functional requirements 

APPLICATION or are known in the art as "protocols/* The implemen- 
tation of protocols is necessary in order to bring order, and 

This is a continuation in part of the patent application of standardization, to the communications field and allow 

Daniel L. Schoo, et al., Ser. No. 08/486,591 filed Jun. 7, equipment of diverse manufacturers to be interoperable. 

1995, pending, the contents of which are incorporated by 1Q Some protocols are considered low level transmission media 

reference herein. related modulation protocols, such as modulation schemes 

implemented in a modem, for example V.34, V.22 bis, etc. 

BACKGROUND OF THE INVENTION other protocols arc considered higher level, and relate to 

A Field of the Invention SUCQ features as error control, transmission control protocols 

This invention relates to the field of data communication n *" d ° ctw °* ^ ™J lin S and encapsulation of data. The 

and apparatus and methods for receiving and transmitting reqmrements of these latter protocols are typically prepared 

data, video and/or audio communications through a com- a u s a ***** Comment document, circulated among 

„ ■ . , . , t j , , the industry, and eventually adopted by the standards bodies, 

munications access system, such as an integrated network _ *" . . . J r 

n ~*»** ™™, a*™ • , . The present invention is concerned with the distributed 

access server. More particularly, the invention relates to r . _ . . . . . A t . 

*, j *u j * j • * * * *u - fin processing of these higher level network control protocols, 

apparatus and methods for distributing the processing or 20 *, , & , , , » ^ • « : r» ■ 

... ii.i .ir • i • Examples of such protocols are the Point -to-Point Protocol 

higher level network protocols for a single communication mnm *u o • i r • t « c n ♦ i /ct rn\ j *l 

, f n -f (PPP), the Serial Lme Interface Protocol (SUP), and the 

session m such a system among multiple computing plat- « , . ~ _ „ 4 , /^™\ 

f orms J or o r Real-time Transport Protocol (RTP). 

c , . - , , . . , In order to process communications between the comput- 

Examples of such higher level network protocols which ^ Qn ^ ]Qcal ^ netwQrk afld ^ remQte ters 

aresubjecttod^ connected y[a ^ ^ ^ ^ of lhe 

Point-To-Pom Protocol (PPP), the Serial Lme Interface nigheHevel protocols Lst be performed In me prior art, a 

™ u ( X - 1116 1 Real * Umc J^Tw* Pr0U : C01 single platform at the network interface has been used to 

(RTP). However, the principles and methods of the invention form ^ fai levd ssi It has beeQ 

are applicable to other types of higher level protocols. ^ ^ ^ results ifl mefficieacies and loss of 

As desenbed in detail below, the processing of the pro- particularly when the maximum call capacity in the network 

tocols is distributed in the preferred embodiments such that access server in either the inbound or outbound direction is 

one computing platform, such as the computing platform at approached. 

a local or wide area network gateway, performs part of the In Qm distributed proC essing invention, a dramatic 
processing of the protocol, while another computing 3S mcrease in the efficiency of the call routing process can be 
platform, e.g., the computing platform in a digital signal acn ieved, thereby maximizing call throughput and minimiz- 
processor (DSP) or modem module, performs the remainder ^ overaU caU mrmed time. Hiis result is achieved by 
of the protocol processing. In this manner, the throughput distributing the computationally intensive protocol process- 
and efficiency of the call processing of the network access ^ of higher leve , network pro tocols (such as Point-to-Point 
server is substantially increased. 40 Protocol ( PPP ) processing or RTP) among multiple com- 
B. Description of Related Art puting platforms such as the modems of the network access 
The methods disclosed herein can be performed by an server assigned to each of the sessions. As another example, 
element of communications equipment referred to herein as processing of the protocols may be distributed by perform- 
a "network access server." The network access server is a ing some of the processing in one platform at the gateway 
device that is capable of receiving a plurality of simulta- 45 and performing computationally intensive protocol process- 
neous incoming calls from the Public Switched Telephone ing in the modems. For example, RTP protocols associated 
Network (PSTN) and routing them to a packet switched with audio and video conferencing may be distributed such 
computer network for transmission to a host computer that some processing of RTP is performed in the gateway 
system, or telephone or other device connected to the platform, for example RTCP, IP, UDP and central coordi- 
computer network. The network access server is also 50 nation of RTP streams, while the modem or DSP platform 
capable of handling multiple simultaneous calls from the performs RTP jitter buffering and encapsulation of audio 
computer network and directing them onto a communica- frames with RTP header data. In addition, the modem or 
tions link in the PSTN for transmission to the remote user. DSP platform perform lower level protocol processing, 
The patent to Dale M. Walsh et al., U.S. Pat. No. 5,528, including all audio processing, transcoding, DTMF (dual 
595, entitled MODEM INPUT/OUTPUT SIGNAL PRO- 55 tone multifrequency tone) generation, and echo cancellation. 
CESSING TECHNIQUES which is fully incorporated by Various types of communication devices are placed at the 
reference herein, describes an integrated network access interface between a modem and a computer network, such as 
server suitable for use in the present invention. Such a routers, terminal servers, and modules sometimes referred to 
device has been commercialized widely by 3Com Corpora- as "gateway cards." These devices implement software 
tion (previously U.S. Robotics Corp.) under the trade des- 60 programs that control the inflow and outflow of calls 
ignation Total Control™ Enterprise Network Hub. Network between the modems and the network. These devices are 
access servers similar in functionality, architecture and referred to herein generally as "network interface" devices, 
design are available from other companies, including One layer of the software hierarchy that is run in these 
Ascend Communications, Livingston Enterprises, devices is known in the art as an "application layer". This 
Multitech, and others. The invention is suitable for imple- 65 document makes occasional reference to the terms 
mentation in network access servers from the above "applications", "application layer" and "application soft- 
companies, and other similar devices. ware layer." As used herein, these terms means a commu- 
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nication control and management software layer above the 
protocol stacks in a communication device, the device 
typically placed at the gateway (or interface) between a 
modem and a computer network. 

To better illustrate some of the advantages of the 5 
invention, one embodiment of the invention is described 
herein in which the modem or DSP platform in the network 
access server perform asynchronous High-level Data Link 
Control (HDLC) framing of Point-to-Point Protocol (PPP). 
The protocol could be synchronous HDLC framing of PPP 10 
in other implementations. The modem performs the flag 
sequence, data transparency, and Frame Check Sequence 
(FCS) on each PPP frame. The second protocol processing 
performed in the modem or DSP platform is Serial Line 
Internet Protocol (SUP). " 

In the prior art, when an application software routine at 
the network access server gateway creates a PPP (or SLIP) 
frame, it checks each byte, looking for a byte that is a control 
character. If the application finds a control character, a PPP 
Escape character (or SLIP Escape character) is stuffed into 20 
the data stream. Then, the original control character is 
translated to a transparent character and stuffed into the data 
stream. This usually requires two buffers, because extra 
characters are added. For PPP frames, while the application 
is looking at each byte of the frame, it must also calculate the 25 
FCS. When the application receives a PPP (or SLIP) frame, 
it must do the reverse of the above process. In some network 
access servers, such as the Total Control network access 
server of 3Com Corporation, a large number of modems 
may be active at any one time. This means that the gateway 30 
computing platform in the network access server would be 
doing this process for each of the modems if the prior art 
technique was used. This results in a extremely heavy 
processing load on one computing platform, and introduces 
latencies and delays in the call routing process. These effects 35 
combine to significantly reduce call throughput, particularly 
where a large volume of calls are simultaneously received or 
transmitted through the network access server. This call 
latency and delay is substantially reduced in accordance 
with the methods and system described herein. Similar 40 
improvements are expected in an situation wherein distrib- 
uted processing of other higher level protocols is performed, 
such as RTP, as set forth below. 

SUMMARY OF THE INVENTION 45 

The present invention provides a method for routing 
incoming or outgoing calls between a computer network and 
the PSTN. For incoming calls, the method comprises the 
steps of receiving the incoming calls and directing the calls 50 
to a plurality of modems, distributing the processing of 
protocols for the incoming calls among multiple computing 
platforms in the modems and processing the protocols for 
the incoming calls, and subsequently routing the incoming 
calls to the computer system over a computer network via a 55 
network interface or gateway. For outgoing calls, the calls 
are directed from the computer network to a plurality of 
modems in the network access server. The protocol process- 
ing is distributed among a plurality of computing platforms 
in the modems, and the calls are sent out to a communica- $q 
tions link such as a Tl telephone line. 

In one preferred embodiment of the invention, the method 
is performed in a network access server having a plurality of 
modems for receiving a plurality of incoming calls or 
modulating a plurality of calls onto a communications link. 65 
The computing platforms comprise the data processing 
structures in each modem. 
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One of the embodiments described below provides for the 
distributed processing of the RTP protocol associated with 
real-time video or audio traffic. Part of the processing is 
performed in the computing platform in the network inter- 
face and some of the processing is performed at one or more 
computing platforms in the modem. Distributed processing 
of RTP in this fashion provides a number of distinct 
advantages, both in terms of scaling, efficient use of com- 
puting resources, and delay reduction. 

First, the technique deals with talk spurts from the packet 
switched computer network side (such as a local area 
network or LAN) to be handled in an efficient manner. If the 
entire RTP protocol was implemented on the network inter- 
face platform, it would make the development of a solution 
for a phenomenon known as "talk spurt", i.e., a burst of 
audio data, more difficult and cause scaling problems in a 
high density network access server. One gateway platform 
can support many different high density and general purpose 
DSP cards (for example, cards dedicated at least in part to 
modem functions), each performing a number of functions. 
Using the techniques described herein, hundreds of calls can 
be processed simultaneously in a single network access 
server. 

Second, it uses resources more efficiently, particularly in 
an environment in which a network operating system is run 
on the gateway computing platform, such as Windows NT™ 
or UNIX-based operating systems. Distributed RTP process- 
ing is designed to allow the gateway application to transfer 
the RTP packet through the gateway to the LAN in one 
kernel mode operation. Conversely, since the operating 
system running on the general purpose DSP or modem 
platform is particularly suitable for real time traffic 
processing, it can effectively handle the RTP end-point 
functionality. 

Third, the method provides for reduced end to end delay. 
The invention not only reduces the overall delay in the 
system, it hides the variation in delay inherent in packet 
switched computer network from the end user. Such advan- 
tages are particularly important in real time voice or video 
applications. The method provides for dynamic jitter buff- 
ering in the modem and/or general purpose DSP cards, 
which adapt to delay due to internal processing within the 
network access server as well as delay in the computer 
network traffic. The term "jitter", as used herein, refers to the 
variation in delay in both the packet switched network and 
within the network access server. Dynamic jitter control can 
be accomplished by increasing or decreasing the amount of 
data that is being buffered within the system. This buffering 
mechanism is able to account for, as accurately as possible, 
the actual jitter or variation in delay characteristics of the 
packet switched computer network while shrinking the 
overall delay within the real time processing provided by the 
network access server. Moreover, if all RTP processing were 
done at the gateway platform, a simple jitter buffer would 
still be required at the PSTN interface due to the jitter added 
by the RTP processing itself inside the system. The basic 
need is to take an a synchronous data steam (from the packet 
switched network) and convert it to a synchronous data 
stream for the PSTN carrier system without noticeable 
impact to the user. By implementing a jitter buffering 
mechanism as a close as possible to the synchronous PSTN 
interface (i.e., in a modem or general purpose DSP comput- 
ing platform), the need for a second buffer, as explained 
above, is removed along with the delay that would be 
attributed to this data storage within the system. 

These and many other advantages and features of the 
invention will become more apparent from the following 
detailed description of presently preferred embodiments of 
the invention. 
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BRIEF DESCRIPTION OF THE DRAWINGS FIG. 18 is a block diagram of the data processing archi- 

Presenlly preferred embodiments of the invention will be ,ecture hi S h density modems, showing the division of 

described in conjunction with the drawings, in which like processing between the RISC chips and the DSPs, showing 

reference numerals refer to like elements in the various «■» pneaaag of the POP/UDP and IP headers by the RISC 

views, and in which: * f^/i" mbou ? d ^J 1 "" P 1 ?* 55 '^' £ d RTP/ 

™A , * ■ ■ * , ,. .„ ... , ... UDP/IP processmg by the DSPs for outbound traffic; 

FIG. 1 A is a block diagram illustrating one form in which . f -ui c 

... .. . . , , . 6 FIG. 19A is an illustration of one possible configuration 

the .invention may be implemented; of the DSPs for the high density modem card, showing the 

FIG. IB is a block diagram illustrating the implementa- type of transcoding that may be performed by the DSPs in 

tion of the routing function of FIG. 1A in the operating an internet telephony application based on RTP; and 

system software of a host computer; p IG ig^ & ^ illustration of a more versatile configura- 

F1G. 2 is an illustration of the overall communications tion of the DSPs for the high density modem card, showing 
system in which an alternative form of the invention is the type of tasks that may be performed by the DSPs, in an 
implemented, illustrating the relationship between various embodiment in which the network access server provides for 
call originators, a telephone network, a network access ^ both remote computer network access and Internet tele- 
server, and a host computer system linked to the network phony types of functionality, 
access server via a network; DETAILED DESCRIPTION OF THE 

FIG. 3 is a schematic block diagram of a network access PREFERRED AND ALTERNATIVE 

server; EMBODIMENTS OF THE INVENTION 

FIG. 4 is a flow chart of the distributed processing ^ Overview and Distributed PPP and Slip Processing Embodi- 

procedure; ments 

FIG. 5 is a detailed flow chart of the PPP routine of FIG. Referring to FIG. 1A, the invention may be implemented 

4. in a communications system in which a call originates from 

' FIG. 6 is a graph of the upload throughput through the "compute; such u a PC 20, which sends data via a Data 

network access server of FIG. 3 using prior art processing in 25 ^—cations Equipment (DCE) (such as modem M) 

. . . , . r r • nnn onto telephone network 50 or other communications link to 

a single platform at the network gateway for performing PPP a qce 10, such as a modem. The call originating 

processing, ( j ata terminal 20 has communication software that uses a 

FIG. 7 is a graph of the upload throughput through the communications protocol, such as PPP or SLIP. The DCE 10 

network access server of FIG. 3 when the distributed pro- demodulates the call from the personal computer 20 and 

cessing technique according to the teaching of the present 30 passes it over a transmission means 11, for example an RS 

invention is used; 232 cable or packet bus, to a router or terminal server 12. 

FIG. 8 is a graph of the download throughput through the The router or terminal server 12 passes the call onto a 

network access server of FIG. 3 when the PPP processing is network or host computer system for example a personal 

performed according to the prior art technique at a single computer (not shown in FIG. 1A). Up to n DCE's 10 may 

platform at the network gateway; 35 be provided, depending on the amount of expected incoming 

FIG. 9 is a graph of the download throughput through the traffic. Or, for some applications, only a single DCE or 

network access server of FIG. 3 when the distributed pro- modem 10 may be required. The DCE 10 and router 12 

cessing technique according to the teachings of the present functions may be implemented in physically distinct 

invention is used - hardware, or combined into a single piece of equipment such 

FIG. 10 is a block diagram of a high density network 40 as ™ the case of network access server discussed in 

access server which may be used to perform the distributed conjunction with FIG. 2. 

processing of the present invention; In a Preferred form of the present invention, the byte to 

FIG. 11 is a dkgram illustrating the distribution of tunc- byte transparency translation processing required by the PPP 

tionality performed by the computing platform in the high (or SLIP) protocol is distributed to the DCE 10 processor, 

density modems cards in the embodiment of FIG. 10, and the 45 rathcr man bemg performed in a computing platform in the 

functionality provided by the computing platform in the ro ^r or terminal server 12 as in the prior art. Tins frees the 

gateway or EdgeServer™ card; application processor (such as the computing platform at 

nr n . . , tJ . r »u j*u*j router or terminal server 12) from this time consuming task. 

FIG. 12 is a high level diagram of the distributed pro- , « n \ , , , . 4 . A * 

u-* c .u Tv£r> * i • *u . i The modem 10 processor already deals with the data on a 

cessing architecture for the RTP protocol in the network - 4 v\ # f tU a > 

« j • A r ,™ 1 ft 50 byte by byte process as it passes data to or from the modem s 

access server embodiment of FIG. 10; ' rZ , CA 4 t , cno 

• i-ii,. a. «. _ data pump. The additional task of data transparency and FCS 

nG ' ■ 13 *. t detaile ? ■ d "? Ua ° f , the distributed calculating is not as burdensome as it is for a multi-session 

t "° C ^ ing ° { ^ RTP pr0 ' 0C01 10 ,he . netW °? aC "f ' application, and hence the modem processor is a preferred 

of FIG. 10, showing the processing performed at the com puti ng .pl a tform.in which to implement. the. invention. 

EdgeServer platform and the processmg performed in the S5 | fa our prefcfred fo ro 0 fihF invention, the appHpUoir> 

modem platform, software processor at the network interface communicates 

FIG. 14 is a illustration of the layers of the S-Bus driver In^mb^nis-lO^via' a.packetibus, but me^mmumca— . 

interface controlling the transmission of messages and data Ubn^l^ C 

between the EdgeServer card and the modem card; cable^^S^ them/such^ > 

FIG. 15 is an overall block diagram of the hardware for 60 4fp||^ Other me T s-~ \ 

the high density modem cards of FIG. 10; J^^^^^^l^ic& to this implementation, ^dl ^ 

FIG. 16 is a diagram of the software architecture for the / mel^afes have a status field and_a lengthy field. There is 

high density modems; exactly'one'PPP-orSLIP ffamTtransmitted or received per 

FIG. 17 is a block diagram of the control processing each packet bus data message. The length field holds the 

architecture for the high density modems, showing the 65 exact number of untranslated data bytes. This is how the 

division of processing between the RISC chips and the modem knows where to insert the flag sequence on outgoing 

DSPs; frames, 
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The basic design of the PPP/SLIP modem 10 is for the particularly important. For example, the router function may 

gateway application software at the router/terminal server 12 exist as a software feature in an operating system 14 for a 

to issue a special command to the modem 10 that puts the personal computer, for example, the various versions of the 

modem into PPP mode, SLIP mode, or neither. Other Windows® program of Microsoft Corporation. In this 

configuration commands that can be issued include: local 5 example, the protocol processing is performed in the modem 

Async-Control Character-Map, remote Async- Control- 10. By virtue of the distribution of the protocol processing 

Character-Map local Delete character translation (i.e. ASCII to the computing platform in the modem 10, rather than in 

7F hex), remote Delete character translation, maximum the CPU for the computer, the computational load on the 

frame size, frame timeout, inter-character timeout, and FCS CPU and the latency in the call routing process are reduced, 

type (e.g. CC1TT 16 bit CRC). The modem 10 informs the 10 providing for increased call throughput, 
application software when it is successfully configured. Additionally, the particular means for transmitting the 

When the modem 10 receives a data message from the calls from the modem to the gateway is not particularly 

application layer for transmission to the remote PC 20, the important. An RS 232 cable, a packet bus, or the internal bus 

modem 10 knows how much data is in the message from the of a host personal computer (such as the ISA, EISA, PCI or 

length field. The modem then creates a PPP or SLIP frame 15 a VESA bus) may be used. While the above examples refer 

by transmitting a frame end character, transmitting the data to SLIP and PPP processing in the modem computing 

(translating as required), calculating and transmitting the platform (or other applicable platform such ISDN TA or 

FCS if it is a PPP frame, then transmitting another frame end CSU), the invention is applicable to other higher level 

character. Note that the PPP address and control fields protocols. 

should be prepended to the PPP data when passed to the 20 Representative Network Access Server Implementation for 

modem because the modem does not interpret any data. PPP or Slip Distributed Processing 

When the modem 10 receives a PPP or SLIP frame from The present invention may also be implemented in a 

the remote location, the modem 10 searches for start of the communication processing system that is depicted generally 

frame character (which is also the frame end character). This in FIG. 2. A plurality of call originators 20, 22 and 24 are 

character is discarded. The modem then examines each 25 located at various remote locations, which transmit incom- 

incoming byte of data, translating transparent data as ing communications to a network access server 30. Call 

required and keeping a current FCS if it is a PPP frame, originator 20, 22 and 24 may consist of a personal computers 

while looking for the trailing frame end character. The CI, C2 and C3, respectively, that generates digital data and 

trailing frame end character is also discarded. If the modem transmits the data to a modems Ml, M2, and M3, which 

is configured for PPP, the current FCS value is confirmed to 30 modulates the data onto a telephone lines 40, 42 and 44, 

be valid and the last two (or four) data characters, which are respectively. For the purpose of this specification, the par- 

the FCS characters, are also discarded (i.e., the length is ticular type of call originators is not important, and the call 

decremented). This frame of raw data (prepended by the originators of FIG. 2 are chosen for purposes of illustration 

address and control fields if PPP) is immediately passed to only. The call originators have communication software that 

the application layer in the gateway (such as the router/ 35 uses the PPP or SLIP protocol. 

terminal server 12) via a packet bus or an equivalent means. In FIG. 2, the data that is transmitted onto the telephone 

Status values indicate OK, invalid FCS, frame too large, lines at 40, 42 and 44 is in analog form. The illustration in 

inter-char timeout, frame timeout, and parity error (frame FIG. 2 assumes that the communication system makes use of 

aborts are discarded and the application layer is not the public switched telephone network (PSTN) 50 such as 

informed). 40 the Tl network. The calls from the call originators are 

Note that, using the techniques of the present invention, digitized and placed into one of the 24 multiplexed channels 

the application computing platform at the network interface of the four-wire Tl span line 51 by the telephone company 

never sees any transparent data. It deals with the actual and fed into the network access server 30, As used herein, 

information in the PPP or SLIP frame. Also note that the the term Tl span line refers to twenty -four 64 kbps 

modem does not interpret the contents of any PPP or SLIP 45 (thousand bit per second) DS0 channels that are multiplexed 

frame. It only does the most basic encapsulation of a in the 1.5444 Mbps DS1 rate, with each DSO channel 

datagram over a serial link, i.e., byte by byte transparency carrying the digital representation of an analog voice chan- 

translation and FCS. The PPP address and control field and nel. The term "trunk," as used herein, refers to a single DSO 

any negotiation, such as LCP or IPCP, are handled at the channel. 

application software layer. Accordingly, by using the tech- 50 The digital signals representing the incoming communi- 

niques of the present invention, data through-put in both the cations are fed into the network access server 30 by the Tl 

uploading and downloading directions through the modem span line 51 (or possibly two span lines). The network access 

10 and routerAerminal server 12 is greatly improved. This is server 30 then routes the incoming call onto the network 52. 

a result of both reduced processing requirements at the The network may be a Token ring network, Ethernet, 

gateway computing platform, and a lower overall latency of 55 Internet, or other type of network, the particular details of 

the system. which are not important. The host computer system 60 then 

Referring again to FIG. 1A, it will be understood that receives the call and process the calls as needed. The host 

other analogous and functionally equivalent DCE's may be computer system 60 depicted in FIG. 2 consists of a variety 

used in accordance with the invention besides the modem of computers such as a personal computer C5, data storage 

10. For example, other computing platforms, such as a those 60 terminal C4, and mainframe computer MC3. As was the case 

in a DSU (data synchronizer unit) and CSU (circuit switch- with the call originators 20, 22 and 24, the details of the host 

ing unit) may be used in conjunction with a digital data computer system 60 has the capability of sending out calls 

network 50. An ISDN (integrated services digital network) via the network access server 30 to the remote data terminals 

terminal adapter may also be used where the calls come in (such as call originator 20). 

via ISDN lines. 65 Referring now to FIG. 3, the network access server 30 is 

Referring to FIG. IB, it will be appreciated that the shown in a functional block diagram form and will be 

physical location of the router or terminal server 12 is not described in more detail. Adescription of the network access 
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server is set forth in the patent of Dale Walsh et al., U.S. Pat. 
No. 5,528,595, entitled MODEM INPUT/OUTPUT SIG- 
NAL PROCESSING TECHNIQUES and the reader is 
directed to the '595 patent for a more exhaustive description. 
The network access server 30 has a chassis 70 which houses 
a telephone interface unit 72 which receives the incoming 
calls on Tl span line 51, demultiplexes the calls, and routes 
the calls over a high speed time division multiplexed (TDM) 
bus complex 74 to twelve quad modem modules 76 A, 76B, 
etc. Each modem module 76 has four modems (not shown 
in FIG. 3) which demodulate the incoming calls. Thus, if 
there are two Tl span lines 51 incoming to the network 
access server 30, there are 48 modems in all for the 48 DS0 
channels incoming into the network access server 30. The 
connections on the TDM bus complex between the tele- 
phone interface unit and the modems are static or "nailed 
up" connections, and are established on power-up of the 
network access server 30. The TDM bus complex 74 carries 
data back and forth between all of the various modules of the 
network access server 30. A high speed parallel bus is also 
part of bus complex 74 and transmits data and messages in 
packet form between the modem modules after demodula- 
tion and the network gateway module 82, 

Each modem module 76 is provided with a corresponding 
modem network interface module 78 A, 78B, etc. The 
modem network interface modules 78 have four sync/async 
RS 232 ports 80. The RS 232 ports 80 are linked to 
computers of the host computer system and may be used to 
output the calls from the network access server 30 to the host 
computer. 

The network access server 30 also includes a gateway 
application module 82, which functions as a routing and 
processing engine for directing calls from the network 
access server to the local area network 52 and vice versa. A 
representative gateway application module is described in 
the above-referenced patent to Dale Walsh et al., U.S. Pat. 
No. 5,528,595, entitled MODEM INPUT/OUTPUT SIG- 
NAL PROCESSING TECHNIQUES and the reader is 
directed to the Walsh et al. patent for a more detailed 
discussion. Gateway cards for use in a network access server 
are commercially available, such as the NetServer™ and 
EdgeServer™ cards from 3Com Corporation, and such 
devices are available from other manufacturers of network 
access servers. A network management module 86 provides 
management and supervision functions for the network 
access server 30. The reader is directed to the patent of 
Rozman et al., U.S. Pat. No. 5,438,614, which is incorpo- 
rated by reference herein, for a more detailed discussion of 
a modem management system for use in a network access 
server. 

The telephone interface unit 72 of FIG. 3 is described in 
detail in the Walsh et al. 5 595 patent, therefore the reader is 
directed to the patent for a detailed discussion of its con- 
struction and functionality. The card is composed of two 
separate modules, an incoming call interface 105 module 
and an incoming call application 175 module. The interface 
module 105 physically receives the incoming Tl span lines, 
converts the signal in a digital TTL format, and delivers the 
signal to the network application module 175. The interface 
module 105 provides a CSU interface which recovers clock 
signals and data from the incoming Tl signals, and also 
provides the transmission of outgoing digital telephone 
signals representing digital data to line Tl . The application 
module 175 provides framing of recovered Tl data to extract 
the Tl DS0 channel data and then switches the channel data 
twenty four time slots on a TDM bus in complex 74 for 
distribution to the all-digital modems in the modem modules 
76. 



10 



30 



35 



40 



50 



55 



60 



65 



The digital modem modules 76 and 78 are also described 
in the Walsh et al. '595 patent in detail, therefore the reader 
is directed to the Walsh et al. '595 patent for a detailed 
discussion of the modem cards, their architecture and 
function, and their interfaces to the TDM bus and the parallel 
bus on complex 74. The cards are available commercially 
from 3Com Corporation, and similar modem cards are 
available from other manufacturers in the industry. Each 
module 76 contains four modems for a total of 24 modems 
for 6 modem cards. As a result, the network access server 30 
of FIG. 3 can handle a total of 24 simultaneous full duplex 
channels of data communication. If two Tl span lines are 
inputted into telephone interface unit 72 (FIG. 3) then 12 
modem modules may be provided to handle 48 simultaneous 
full duplex channels. Of course, additional capacity may be 
provided if desired. 

The protocol processing for the incoming and outgoing 
calls, in the present embodiment of the invention, is distrib- 
uted among four modem control processors, one per modem, 
in the card 76. The data processing and modem control 
processing functions of the DSP data pump and the modem 
control processor may be combined into a single digital 
signal processor. Additionally, part of the protocol process- 
ing is performed in the computing platform in the gateway 
module 82, as described in detail below. 

Distributed Processing of PPP and SLIP Protocols 

With the above description in mind, the reader's attention 
is directed to FIG. 4 and FIG. 5, which is a flow chart of the 
distributed PPP and SLIP processing procedure according to 
one form of the present invention. The following discussion 
is made in reference to the network access server embodi- 
ment of the invention, and persons of ordinary skill in the art 
will readily understand that the description can be adapted to 
other possible embodiments. 

At step 200, a call is arrived from one of the remote call 
originators, or else a call is initiated from the host computer 
system to a remote computer or other data terminal. A step 
202, the call is answered in a well known manner. At step 
204, the gateway application module 82 (FIG. 3) determines 
the high level protocol of the incoming or outgoing call. This 
is done by determining, for example, the port configuration 
of the call, by the telephone number of the call originator or 
the call destination, by conversation or menu with the user, 
by a user name and/or a database look-up, by an automatic 
detect or inspection of data routine, or whatever other 
process is implemented. 

At step 206 the gateway module 82 configures the pro- 
tocol mode for the modem. The software routine CONFIG. 
l.S set forth in published PCT application serial no, PCT/ 
US96/09661 of Daniel L. Schoo et al. is implemented in this 
step. The above PCT application is incorporated fully by 
reference herein, and references in this document to "soft- 
ware listing" refer to the software listing appended to the 
above PCT application. If the asynchronous transparent 
protocol is performed, at step 214 a forward on timeout or 
buffer full command is sent to the modem in the modems 
module 76. If the SLIP protocol is used, for receiving calls 
at step 210 the modems remove transparency and translation 
characters, and send the payload (a number of bytes) or 
overflow errors to the gateway module 82. In the above - 
referenced software listing, routines in SLIPRX.S are imple- 
mented at this step. For transmitting calls, at step 212 the 
modems add transparency translation characters and a SLIP 
framing character. In the appended software listing, the 
routines in SLIPTX.S are implemented. 

If PPP protocol is used, the routine in CONFIG.l.S is 
implemented at step 208. Step 208 of FIG. 4 is shown in 
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detail FIG. 5. Referring now to FIG. 5, at step 216 the 
gateway module 82 enters the LCP negotiation procedure 
per the instructions set forth in the Request For Comments 
(RFC) 1661 Standard. This is a publicly available document 
which specifies an Internet standards track protocol for the 
Internet community. Persons of ordinary skill in the art are 
familiar with the RFC 1661 document. 

At step 218, the gateway module configures the modems 
for each LCP negotiated parameter. This may be accom- 
plished in order, for example, MTU, TX Async map, RX 
Async map. In the appended software listings, the routine in 
CONFIG2.S describe this procedure. The procedure then 
enters a data transfer phase at step 220 in which "messages" 
are uploaded or downloaded through the network access 
server 30. In the receiving or uploading direction, the 
modems remove the HDLC frame characters, and send the 
payload, number of bytes and error code (such as good FCS, 
bad FCS, overrun, or partial), as illustrated in module 222. 
This procedure is set forth in the software listing PPPRX.S. 

In the transmitting (downloading) direction at step 224, 
the modems get the payload from the gateway module 82, 
perform character transparency translation and transmit 
asynchronous map, calculate the FCS, and add HDLC 
framing characters. At step 226, the messages are sent out 
through the telephone network 51. This procedure is set 
forth in the PPPTX.S routine in the software listing. 

One representative example illustrating the advantages of 
the invention in terms of throughput over the prior art 
methods can be seen if FIGS. 6-9. FIG. 6 is a graph of the 
upload throughput using prior art PPP protocol processing in 
a single platform at the network gateway. Note that the total 
throughput of the network access server remains substan- 
tially constant after a peak is reached with 8 active ports and 
levels out at 17 active ports at 24,000 bytes per second. The 
per port throughput starts off quite high at roughly 4600 
bytes per second for a small number of active ports (that is, 
less than 10), but drops dramatically at higher numbers of 
active ports. Thus, FIG, 6 illustrates the processing "bottle- 
neck" that occurs when the PPP protocol processing is 
performed in a single platform at the network interface. 

FIG. 7 is graph of the upload throughput when the 
distributed processing technique according to the teaching of 
the present invention is used. Note that in FIG. 7 the total 
throughput rises steadily up to 18 ports, dips slightly 
between 18 and 21 ports, and then increases up to total 
throughput of approximately 92,000 bytes per second with 
25 active ports. Between 25 and 48 active ports, the total 
throughput remains between roughly 72,000 bytes per sec- 
ond and 86,000 bytes per second. This is approximately 3 to 
4 times the total throughput as compared to the results of 
FIG. 6. Further improvements in system tuning would 
smooth out the oscillations in system throughput shown in 
FIG. 7, especially for higher numbers of active ports. Note 
also that the average per port throughput using the tech- 
niques of the present invention remains relatively high as the 
number of active ports increases, accomplishing an effi- 
ciency improvement of more than 3 times compared to the 
results of FIG. 6, particularly at higher volumes of active 
ports. 

FIG. 8 is a graph of the download throughput when the 
PPP processing is performed in the prior art technique of a 
single platform at the network gateway. In FIG. 8, note that 
the total download throughput levels off at approximately 
55,00 bytes per second between above 20 active ports when 
the prior art techniques are used. The average per port 
throughput starts at roughly 45,000 bytes per second with 
under 10 active ports, but this value drops steadily to a low 
of approximately 12,000 bytes per second when 48 active 
ports are used. 
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FIG. 9 is a graph of the download throughput when the 
distributed processing technique according to the teachings 
of the present invention is used. Note that the total download 
throughput rises steadily to a high of approximately 104,000 

5 bytes per second when 26 active ports are used and remains 
relatively constant at higher levels of call activity. As can be 
seen by comparison of FIG. 9 to FIG. 8, this is a substantial 
improvement in call throughput. The average per port 
throughput using the techniques of the invention remains 

10 relatively constant at 4500 bytes per second up to 22 ports 
and this value drops gradually to approximately 2000 bytes 
per second when 48 ports are used. This represents an 
improvement of approximately 2 to 1 over the download per 
port throughput, particularly at high call volumes. 

15 Presently preferred software routines for implementing 
the procedures shown in FIG. 4 and FIG. 5 are set forth in 
code form in the Appendix to the above-referenced pub- 
lished PCT application, A person of ordinary skill in the art 
will appreciate that the code represents but one example of 

20 the implementation of the invention in a network access 
server environment. The code is stored and executed in the 
modem DSPs of the modem modules 76 of FIG. 3. The 
software routines in the gateway module 82 that interact 
with the modem software routines comprise straightforward 

25 higher level control routines, with which can be readily 
developed by persons of ordinary skill in the art. 
Distributed Processing of RTP and RTCP Protocols in High 
Density Network Access Server 
An alternative embodiment of the invention, involving 

30 distributed processing of RTP and RTCP, will be described 
below in conjunction with a high density version of a 
network access server. It will of course be readily apparent 
that the distributed processing of RTP and RTCP can be 
performed in the network access server of FIG. 3, and the 

35 PPP and SLIP distributed processing can also be performed 
in the embodiment described below. 

The architecture of the alternative network access server 
30 embodiment is shown in FIG. 10. The embodiment of 
FIG. 10 is a high density network access server, in that each 

40 general purpose DSP or modem card 376A, 376B, etc. 
contains a high density DSP configuration capable of han- 
dling 23, 24 or 30 DS0 channels, instead of four DS0 
channels per modem card in the embodiment of FIG. 3. By 
providing a set of high density modem cards 376 and a 

45 robust computing platform in the network gateway 382, a 
single chassis can process many hundreds of calls through 
the device simultaneously, as compared to 24 or 48 calls in 
the embodiment of FIG. 3. The term "HDM" for the DSP 
cards 376A, 376B etc. in FIG. 10 is an acronym for "high 

50 density modem," indicating that each card performs modem 
functions for a large number of channels on the telephone 
line, such as 23 B channels plus 1 D channel for an ISDN 
Primary Rate Interface, 24 DS0 channels for a Tl line and 
30 channels for an El line. 

55 In the embodiment of FIG. 10, each DSP card 376 has its 
own Tl/ISDN telephone line interface 372 connected to an 
ISDN PRI or Tl line. The Tl/ISDN telephone line interface 
372 is similar in architecture and function of the interface 72 
of FIG. 3 and set forth in the Walsh et al. '595 patent, and 

60 is connected to the high density modem cards by a TDM bus, 
as described in detail in the Walsh et al. '595 patent. An 
alternative would be to provide a plurality of Tl/ISDN 
telephone line interfaces 72 and distribute DS0 channel data 
to the modems via a TDM bus with extra highway lines, as 

65 described above in the embodiment of FIG. 3. 

The high density DSP cards 376 are connected to the 
gateway card 382 via a high speed parallel packet bus 374, 
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similar to that described in the Walsh et al. patent. The 
number of high density DSP cards 376 and associated 
telephone line interface cards 372 is essentially arbitrary, but 
10 to 24 such cards are typical in a high density network 
access server application today, providing modem function- 
ality for between 240 and 576 Tl DSO channels. The HDM 
cards are described in detail below. 

Hie gateway or EdgeServer™ card 382 consists of a 
general purpose computing platform (such as an IBM PC) 
running a stand alone or shareware network operating sys- 
tem such as Windows NT™ from Microsoft Corporation or 
UNIX. The card 382 contains software and hardware mod- 
ules to perform call routing, modem configuration and other 
features as set forth and described for the gateway modules 
in the Walsh et al. '595 patent and the Baum et al. patent, 
U.S. Pat. No. 5,577,105, also incorporated by reference 
herein. Further details on the design and features of the 
EdgeServer™ card 382 are set forth in the patent application 
of William Verthein et al. Ser. No. 08/813,173, the contents 
of which are incorporated by reference herein. 

The network access server 30 shown in FIG. 10 is useful 
for a number of different types of applications, such as 
Internet access, remote access to corporate backbone 
networks, video and audio conferencing, Internet telephony, 
digital wireless Internet and corporate network access, to 
name a few. In an Internet telephony embodiment, the 
product provides a facility for users to engage in long 
distance telephone, audio/visual and/or data sessions using 
the Internet as the transport medium rather than the long 
distance public switched telephone network of the inter 
exchange carriers. Users realize a substantial savings in 
transmission charges as compared to phone charges. 

A standards-track protocol has been develop by the indus- 
try to provide end-to-end delivery for data with real-time 
characteristics, such as audio, video, and simulative data, 
over multicast or unicast network services. This protocol is 
described in detail in a publicly available document known 
as RFC 1889, January 1996, the entire contents of which are 
incorporated by reference herein. In the RTP, the data 
transport is augmented by a control protocol referred to as 
RTCP (Real-time Transport Control Protocol) to allow for 
monitoring of the data delivery in a manner scalable to large 
multicast networks, and to provide control and identification 
functionality. The RTP and RTCP are designed to be inde- 
pendent of the underlying transport and network layers in 
the OSI model. In one representative embodiment of the 
invention described below, the processing of the RTP and 
RTCP protocols for each call passing through the network 
access server 30 is distributed between the gateway or 
EdgeServer card 382 and the DSP platform in the HDM 
cards 376. 

The RTP has header fields that are either fixed or deter- 
ministically varying, with a certain format described in the 
RFC 1889 document. These fields include fields for a 
sequence number, timestamp, synchronization source 
identifiers, and contributing source identifiers. The header 
can be extended with RTP header extension to provide new 
payload-format-independent functions that require addi- 
tional information to be carried in the RTP data packet 
header. Additionally, the RTCP packets have a packet format 
and header fields for control information. 

The network access server 30 of FIG. 10 is an ideal 
platform in which to direct calls between remote and net- 
work endpoints of a call using RTP The software and 
hardware functionality of the HDM DSP platform and the 
EdgeServer platform is distributed in the manner shown in 
FIG. 11. The telephone line interface cards 372 of FIG. 10 
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perform PSTN access functionality, including Tl signaling, 
call supervision (E&M, Loop Start, Ground Start), time slot 
routing and Tl framing, ISDN signaling and B -channel 
routing. The HDM cards 376 perform information flow 
processing, including tagging and delivery of audio and 
control components, a portion of the processing of the RTP 
(described below), audio transcoding in accordance with the 

G. 771 standard (or G.723 G.729), and other audio process- 
ing functions (e.g., DTMF generation, echo cancellation, 
etc.). The EdgeServer 382 or gateway platform performs call 
management, a portion of the RTP processing (described 
below), H.245 processing, H.225 processing, Internet Pro- 
tocol (IP) routing, and gateway to HDM coordination pro- 
tocol processing. 

The arrows above the blocks in FIG. 11 indicate specific 
data forms for an Internet telephony session that are directed 
between the telephone line interface card 372, HDM DSP 
card 376, and EdgeServer card 382 for both inbound and 
outbound data. For data going between the packet switched 
computer network and the PSTN (referred to herein as 
outbound data), asynchronous H.323 over IP data is received 
at the EdgeServer card 382. Audio frames/Protocol Data 
Units (PDUs) and modem control information is passed in 
packet form from the EdgeServer card to the high density 
modem card via the packet bus 374 (FIG. 10). The HDM 
DSP platform in the card 376 performs additional RTP 
processing and other audio processing described herein, 
including jitter buffering, G.711, G.723 and G.729 
transcoding, and Tl/ISDN signaling. The data is sent in 
PCM (pulse code modulated) form from the HDM card to 
the telephone line interface 372 over the TDM bus (FIG. 10), 
framed into Tl or ISDN PRI frames, and sent on the PSTN 
for transmission to the destination. 

FIG. 12 is an overview illustration of a presently preferred 
distributed RTP processing architecture for the network 
access server embodiment of FIG. 10. Sources of audio/ 
video or other real time data, such as a plurality of H.323 
personal computers, are located on the packet switched LAN 
or WAN connected to the network access server. The data 
from the H.323 PCs comes into the EdgeServer cards as 
LAN traffic (over IP). 

In one embodiment, the EdgeServer card 382 handles all 
RTP functionality, with the exception of jitter buffer and all 
frame-based audio processing and generation of RTCP send 
and receive reports, which are performed in the HDM card 
376 platforms. The actual processing performed by the 
EdgeServer 382 includes the handling of the creation and 
destruction of the audio streams as directed by the H.245 and 

H. 225 protocol. Additionally, the gateway or EdgeServer 
card 382 acts as a central coordinator for the RTP streams 
and provides the HDMs with all the information that they 
need to provide their part of distributed RTP functionality. 
(In one possible variation, the gateway card 382 also termi- 
nates all RTCP traffic, including termination and generation 
of sender reports and receive reports. IP, UDP and RTP 
headers are placed onto all audio frames sent to the LAN. 
The EdgeServer 382 fills in all RTP header fields that change 
on a dynamic basis.) The EdgeServer also provides a trans- 
lation function for software interfaces between the 
EdgeServer card 382 and the LAN interface, and between 
the card 382 and the packet bus 374. In one possible 
variation, the RTCP processing is distributed between the 
computing platform in the card 382 and the HDM card 376 
platform, such that information for the send and receive 
reports for RTCP comes from the HDM platform. The actual 
send and receive reports may be sent out by either the HDM 
modem platform in the cards 376 or by the computing 
platform in the gateway card 382. 
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Still referring to FIG. 12, the HDM 376 digital signal 
processor computing platform handles the processing of the 
RTP jitter buffer and all lower level audio processing that 
occurs below this level, including DTMF generation, echo 
cancellation and transcoding. The actual processing that is 
encompassed within the HDM platform includes a coordi- 
nation function with the Edgeserver card to respond to its 
setting up new RTP session, or tearing down old sessions; 
providing a jitter buffer to remove any arrival jitter; perform 
all RTP header encapsulation and de -encapsulation func- 
tions; perform all IP and UDP header encapsulation and 
de-encapsulation functions; and reacting to changing 
Internet/Intranet conditions by changing the size of the jitter 
buffer. 

An embodiment of distributed processing of RTP func- 
tionality between the EdgeServer and the HDM cards is 
shown in more detail in FIG. 13. The left hand margin shows 
the form in which data (such as audio data) is transmitted 
between the user H.323 client on the computer network and 
the PSTN phone client linked to the network access server 
via the PSTN. As can be seen in the Figure, audio is sent 
encapsulated in an RTP header from the H.323 client all the 
way into the HDM platform 376. The HDM platform 376 
removes the RTP headers and performs jitter buffering in a 
jitter buffer, with the size of the jitter buffer dynamically 
changed in order to deal with the bursty, asynchronous 
nature of packet switched data from the computer network. 
The modem platform 376 also performs lower level DSP 
processing of audio frames, including transcoding, echo 
cancellation, and DTMF generation. The audio data is 30 
transcoded according to the G.711 standard and sent over the 
PSTN to the phone client. 

Still referring to FIG. 13, the outbound data from the LAN 
is translated through a network interface software structure 
302 (WinSock, BSD sockets or TDI), the details of which 
are not important aad readily derived by persons of skill in 
the art. A receive RTP packet transfer module 304 operates 
with a Calculate RTP Receiver Reports module 306 and 
advises the user H.323 client of the number of lost packets, 
size of the jitter, and other information per RFC 1889. The 
audio data encapsulated in RTP header is translated by the 
S*Bus interface software structure 308 (described below) 
and sent over the packet bus 374 (FIG. 10) to the HDM 
modem cards 376. 

A reduced instruction set central processing unit (RISC) 
computing platform in the HDM card implements a similar 
S-bus interface software program 310 and receives the RTP 
audio data, and performs the jitter buffering (described 
below). The RISC computing platform 312 calculates the 
needed size of the jitter buffer based on LAN/WAN condi- 
tions and any intrachassis latency. The RISC computing 
platform 312 removes all RTP headers and passes audio 
frames to the DSP computing platform 314 in the HDM 
card. The DSP platform performs other lower level audio 
processing, including DTMF generation, echo cancellation, 
and ultimately the G.711, G.723 and G.729 transcoding. The 
inbound processing is as described in FIG. 13, with the 
encapsulating of audio framing with RTP headers performed 
by module 316. 

While the embodiment of FIG. 13 shows the processing 
of RTCP send and receive reports on the EdgeServer card 
382, it may readily be shifted to the high density modem 
platform in the HDM card 376. For example, the RTCP 
protocol processing may be distributed among the gateway 
platform and the DSP platform in the HDM card, for 
example the information for the RTCP send and receive 
reports comes from the modem platform but the actual send 



and receive report may be generated and sent out by either 
the DSP platform in the HDM card or at the gateway card 
382. 

Referring now to FIG. 14, the software architecture for 
the S-BUS driver interface for the EdgeServer card 382 is 
illustrated in block diagram form. The interface consists of 
four driver layers dynamically linked together: application 
drivers (system control, Q.931 messaging, Voice Prompt, 
H.323 interworking, and Real-time Audio); a message router 
driver 352, an S-BUS messaging driver 354 and a packet bus 
(PB)-PCI driver 356. 

All application drivers 350 receive and transmit their 
message via the message router driver. The message router 
driver is the holding place for various message types such as 
audio, data and control, which are queued into three corre- 
sponding transmit or receive priority queues. The system 
control driver interfaces with the message router driver to 
send requests and receive events to and from the SBUS 
message driver 354. The system control driver acts as a 
module-to-module communication for the underlying 
EdgeServer components. The module-to-module communi- 
cations include the following types of messages: 

Call establishments — Call establishment messages are 
used to signal the HDM or System Control of various 
call establishment requests and responses. For 
example, Dial-out, Answer Call, Disconnect Call, Con- 
nection Failed/Succeeded, and Incoming Call. 
Call maintenance — Call maintenance messages are used 
by the System Control to request HDM's statistics in 
real-time. Likewise, the HDM responds back to the 
System Control with the appropriate message. 
Resource management — Resource management mes- 
sages are used by the System Control to request various 
HDM's resource information. For example, availability 
of HDMs and its number of ports/channels, DSPs 
utilization, and its number of DSPs available for 
transcoding. Likewise, the HDM responds back with 
the appropriate resource information. 
Configuration — Configuration messages are used by the 
System Control to send configuration messages to an 
HDM to add and/or modify the flavor of a connection. 
For example, Enabling/Disabling HDM Ports. 
Diagnostic — Diagnostic messages are used by the System 
Control to request a specific test to be executed at the 
HDM level in conjunction with the associated 
EdgeServer' s test code. For example, various level of 
loopbacks at the HDM's hardware components, client's 
terminal loopback, and measuring round-trip delay. 
The Q.931 Driver 358 is responsible for sending Q.931 
50 messages to HDM and processing receive Q.931 messages 
from the HDM. Q.931 is a message-oriented control proto- 
col for call setup/tear-down in the ISDN environment. All 
Q.931 messages are control messages which will be pro- 
cessed by this driver and the HDM. 
55 Voice Prompt driver 360 is an optional driver and man- 
ages the record, store and play back of voice prompts. 

The H.323 Interworking Driver 362 interfaces to the 
Message Router Driver to send requests and receive events 
to and from the SBUS Message Driver. The H.323 Inter- 
go working Driver relationship with the SBUS drivers is to 
communication the following message type between the 
EdgeServer and HDM: 
Audio Mode Exchange with HDM. 
Open an Audio path between SBUS drivers and Real-time 
65 Audio Transfer Driver. 

All H323 interworking messages are control messages 
which will be processed by this driver and the HDM or 
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SBUS drivers. The Real-time Audio Driver 364 sends and In the illustrated embodiment, one HDM card support 24 

receives packetized audio via SBUS Messaging driver. calls (i.e., a Tl span). Therefore the HDM architecture is 

Audio packets are enqueued and dequeued in the audio based on the HDM 24/30 hardware architecture consisting 

queue of the Message Router 352. The Message Router Q f 12 high performance Texas Instruments TMS3201c548 

Driver 352 is an intermediate driver in kernel-mode which 5 d$Ps and 3 IBM PPC403GCx RISC processors. An alter- 

connects to the Application drivers above it and the SBUS nativc embodiment for a 30 channel carrier, such as El, 

Messaging Driver 354 below it. The Message Router 352 would mc i u d e a daughter board which supports four addi- 

provides the interface to support the various multimedia tional c54g Dsp thcreb providing somc redundancy m 

message types (audio, data, and control) that are communi- ^ D gp en£mes 

cated between the Application drivers on the EdgeServer „ ™ um ?. * , ClU , , . . , 

and the HDMs. Each multimedia message that is received 10 . ™ e H ™ 15 * D ™> u / 10n °* ihc f dl ^ tal mo6 ™ tcC * n ° l °- 

from the HDM, the SBUS Messaging component parses the P es foun , d m Walsh et al patent providing a highly 

packet's multimedia header to distinguish its priority level integrated processing solution for purely digital configura- 

and pass this message to the appropriate Message Router's Uons of a network access server - The HDM does not support 

priority queue. From this queue this driver passes the RS-232 and POTS interfaces. The existing HDM 24/30 will 

multimedia message to the appropriate Application Driver 15 terminate both analog modulations, via A-law or u-law PCM 

350 based on the application ID in the multimedia header. encoded data, and ISDN calls such as V.120, X.75, or Async 

Likewise, the Message Router 352 transmits the multimedia to Sync PPP at 64 Kbps/56 Kbps or V.110 from 300 bps to 

messages from the Application Drivers 350 to the SBUS 38.4 Kbps. 

Messaging's 354 transmit priority queues. The Message As noted above, the HDM 24/30 NAC is designed to 
Router 354 communicates to the HDM using the hardware 20 interface to the telephone line interface card that provides a 
and data link services of the SBUS drivers. direct interface to either a Tl or El span. There are two types 
The PB-PCI driver component 356 is a Windows NT™ of Tl signaling functions supported in the HDM: channel- 
driver which interfaces with the packet bus hardware, for the iz e d Tl and PRL For channelized Tl , the HDM supports 24 
purpose of data delivery between the EdgeServer, HDM and DS0 channels; for PRI, the HDM supports 23 B-channels 
other cards on the network access server 30. Ihe SBUS 25 and x D-channel; for El the HDM supports 30 B channels 
Messaging component utilizes the PB-PCI driver as a physi- an d i D-channel 

cal level device to transmit and receive messages between FIG. 16 is a block diagram of the software architecture of 

FTO^f : HhS ™^todkdia r of tb h' 't the ™ M 3?6 ' ^ S ° ftWare ' S deSifiQed t0 SUpP ° rt b0th ^ 

j ' j a TTMi * . _r ion -j • * control processing and data processing. A packet bus driver 

modem cards 376. A TDM interface 380 provides an inter- , , *, ftn . 4 ? ..,,{* , fi , „ Tr , inx 

face to a TDM bus leading to the Tl/El interface card (see 30 rootle 400 interfaces with the packet bus 374 (see FIG. 10) 

FIG. 10). The details of the TDM bus are known in the art, t0 reoeive and transmi1 P ackets of dala between tne HDM 
and the reader is directed to the similar interface in the Walsh and mc gateway or EdgeServer card. A Packet bus S-BUS 
et al. patent for the quad modem cards. The DS0 channel messaging routine resides above the packet bus driver and 
data is sent from the TDM interface 380 to a DSP subsystem performs translation of S-BUS message fields from the 
382, comprising 12 high performance Texas Instruments 35 EdgeServer. Control and signaling data is sent to the Board 
TMS3201c548 digital signal processors (DSPs) 384, each Manager call control module. Signaling and call control for 
DSP supporting bidirectional modem functionality for two telephony and ISDN calls are separated and sent to a 
separate DS0 channels. The DSP subsystem 382 is con- signaling and control manager 406, which supervises a 
nected by a bus system 386 to an IBM PPC403GCx reduced DTMF or other tone generator 408, a Tl signaling-call 
instruction set central processing unit (RISC) 388 function- 40 supervision manager 410, and an ISDN Q.921/Q.931 sig- 
ing as a board manager, and to a shared memory 390. The naling manager 412. The signaling and call control data is 
shared memory 390 is connected to two additional IBM sent to a PCM signal interpreterAranscoding module for 
PPC403GCx RISC application co-processors processors required signal conversion into DS0 channel data. 
392Aand 392B, and to a packet bus interface 394. The RISC In the HDM, audio data is passed from the Packet Bus 
co-processors 392 perform the functions illustrated in FIG. 45 S-BUS messaging module 402 to a dynamic jitter processing 
13. The hardware details of the packet bus interface 394 are module 420 for removing jitter in the audio stream. Aboard 
conventional and described in the Walsh et al. patent. manager data path module 422 supervises the jitter 
The following summarizes some of the major functions processing, RTP header processing, and audio packetization 
for the network access server performed by the high density and depackatization functions, performed by modules 420, 
modem card 376: so 424 and 426, respectively. An audio transcoder module 428 
ISDN Signaling resides below the audio packetizing and depacketizing mod- 
PSTN (Channelized Tl) Signaling ule for performing G.711 transcoding. Below the audio 
Resource Management transcoder, an echo cancellation module 430 and an auto- 
Data Path Setup and Thread Management matic gain control (AGC) module 432 perform lower level 
Flow Control ss processing. The audio data is passed to the PCM signal 
Packet Bus/S-Bus Processing interpreter module 414 and forwarded to the Tl/ISDN 
RTP/UDP/IP Header Processing telephone line interface 372 (FIG. 10) as DS0 data for 
Jitter Estimation and Dejittering framing and transmission on the telephone line to the 
Packet Sequence Re-ordering destination. 

Audio Packetization 60 The call control processing performed by the HDM is 

Audio Transcoding shown in FIG, 17. The 403GCX RISC chip platform 388, 

Echo Cancellation 392A, B (FIG. 15) performs the required S-BUS/Packet Bus 

Automatic Gain Control processing 400, 402 for inbound and outbound data between 

DTMF Tone Generation the HDM and the EdgeServer, the board manager call 

Most of the above functions are conventional and well 65 control module 404, the telephony/ISDN signaling and 

known in the art. The functions relating to RTP processing control processing as indicated in the Figure. The DSP 

are described in detail below. platform performs the DTMS/MF tone generation and call 
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signaling tone detection 438. The DSP platform also imple- 
ments the PCM signal interpreter module 414. 

Referring to FIG. 18, the HDM data processing for the 
HDM is shown in block diagram form. As can be seen, the 
distribution of functionality between the RISC processing 
and the DSP processing is slightly different for inbound and 
outbound data. For inbound data, the DSP performs pro- 
cessing required at the TDM interface, AGC, echo cancel- 
lation and audio transcoding. Audio packetization and the 
addition of RTP/UDP/IP headers for the data is performed 
by the RISC chips, as is the S-BUS and Packet Bus 
processing. 

For outbound data, after the S-BUS and Packet Bus 
processing by the RISC chips, the DSP assumes the task of 
performing jitter processing, removing RTP/UDP/IP 
headers, and audio depacketization, and the lower functions 
of audio transcoding, echo cancellation and TDM interface 
processing to produce DSO channel data for the respective 
Tl/ISDN network interface card. 

The major resource in the HDM required for supporting 
internet telephony in a network access server is the DSP 
transcoding resource. Based on the system requirements, it 
is possible to deliver 3 full duplex G.723/G.711 transcoders 
in each Texas Instruments C548 DSP engine. As a minimum, 
each C548 DSP (12 per HDM card) is required to handle two 
full-duplex audio transcoding tasks. FIG. 19A illustrates a 
HDM/DSP subsystem configuration for such a minimum 
configuration. 

To support an embodiment providing both remote access 
services and Internet telephony applications, a more flexible 
and extensible architecture is required. A new DSP archi- 
tecture can be designed with the goal of creating an exten- 
sible architecture which can be migrated to support addi- 
tional applications and features without significant 
modifications to the architecture. In such a system, an 
event-driven DSP architecture that is flexible and robust is 
preferred, but which fixes the functions that will be sup- 
ported at the compile time. The DSP resources might be 
allocated to V.34 tasks, audio CODEC tasks, or any com- 
bination that is allowed by the available resources. A pro- 
posed event-driven architecture that allows each DSP to 
perform a mixture type of tasks with the goal to fully utilize 
the processing power of each DSP engine is shown in FIG. 
19B. Ideally, in such an architecture, the goal is to produce 
an architecture with an operating system that allows multiple 
functions to run and modularize functionality so that one can 
modify/add/delete modules with very little modification to 
the operating system. 

RTP/UDP/IP Header Processing Module 424 

The RTP/UDP/IP header processing is supported by the 
application processing layer in the RISC. The RTP/UDP/IP 
header processing module is responsible for adding the 12 
octets of RTP header, 12 octets of UDP header, and 20 octets 
of IP header to all the audio packets which are transmitted 
from the PSTN to the LAN. This module is also responsible 
for removing the RTP/UDP/IP header from the audio pack- 
ets for data from LAN to the PSTN. The RISC application 
co-processors also perform PPP/SLIP framing, as well as the 
RTP/UDP/IP processing. 
G.723.1/G.711 Audio Transcoding Module 428 

The audio transcoding between G. 723.1 and G.711 is 
required for the DSP subsystem since the G.711 audio 
stream will always arrive from the PSTN/ISDN clients over 
Tl and the G.723 compressed stream arrive from the LAN 
side. 

Specifically, the following functions are supported by the 
DSP subsystem: 
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Each DSP engine supports 2 concurrent full-duplex 

G.723. 1/G.711 transcoder tasks. 
The transcoder implementation is compliant to the 
G.723. 1 and G.711 ITU implementation. 
5 It supports encoder/decoder independence such that one 
can allocate any combination of encoders or decoders 
according to system configuration and within the DSP 
resource limits. 
It supports synchronous output to the Tl interface. 
10 Jitter Processing Module 420 and Packet Re-Ordering Mod- 
ule 426 

Jitter (as defined previously) can be caused by network 
congestion external to the network access server 30, or the 
network access server's internal communication and pro- 
15 cessing. The network congestion may also cause the packets 
to arrive in a wrong order, a phenomenon termed a packet 
sequence disorder. These conditions are ameliorated by the 
jitter processing module 420 and the packet re-ordering 
module 426. 

The DSP subsystem 382 supports a dynamic jitter pro- 
cessing module 420 where the size of the jitter queue is 
adjusted during the run time operation. This module 420 will 
also support the re-ordering of the frame sequence. In 
addition, the module will support the talk-spurts processing 

25 to reduce the bandwidth usage in the LAN. 

The RISC application co-processors 392 support an audio 
packetization function 426. This audio packetization func- 
tion conform to the ITU H.225 Annex F — new audio pack- 
etization for G.723. 1. Both 6.3 kbps and 5.3 kbps rates are 

30 mandatory part of the G.723. 1 encoder and decoder. A 
G.723. 1 frame can be one of three sizes: 24 bytes, 20 bytes, 
or 4 bytes. These 4-byte frames are called SID (silence 
insertion descriptor) and are used to specify comfort noise 
parameters. There is no restriction on how 4, 20, and 24 

35 bytes are intermixed. The first two bits in the frame deter- 
mine the frame boundary. It is possible to switch between 
the two rates at any 30 ms frame boundary. This packetiza- 
tion scheme is compliant to RFC 1890 for the packetization 
interval with the following specification: 

1. The first packet of a talk-spurt (first packet after a 
silence period) is distinguished by setting the market bit 
in the RTP data header. 

2. The sampling frequency (RTP clock frequency) is 8000 
45 Hz. 

3. The packetization interval should have a duration of 30 
ms (one frame) as opposed to the default packetization 
of 20 ms 

4. Codecs should be able to encode and decode several 
50 consecutive frames within a single packet. 

5. A receiver should accept packets representing between 
0 and 180 ms of audio data as opposed to the default of 
0 and 200 ms. 

Echo Cancellation Module 430 

ss The DSP subsystem 382 supports a line echo cancellation 
(LEC) module 430 which complies the performance require- 
ments of ITU G.165 except portion of tone disabler. The 
tone disabler defined by G.165 is primarily used to guard 
tones in telephone networks and can be implemented in the 

60 system when it is necessary. The echo cancellation uses 
signal correlation techniques to determine parameters of a 
filter that processes the incoming signal on the 4-wire side 
of a hybrid. The filter forms an estimate of the echo when an 
incoming signal is present. This estimate is subtracted from 

65 the signal on the return path. 

Presently preferred embodiments have been set forth 
above. Persons of skill in the art will appreciate that modi- 
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fications may be made from the disclosed embodiments form performs a first portion of the processing of said 
without departure from the spirit and scope of the invention. higher level protocol and said modem computing plat- 
For example, the assignment of processing to the RISC form in said modem performs a second portion of the 
and/or DSP platforms is a matter of design choice and may processing of said higher level protocol; and 
vary from the embodiments set forth above. Further, the s subsequently directing the data from said modem corn- 
processing of the high level protocols can be distributed puting platform through said telephone line interface 
between the gateway and the modem platforms in a manner onto said telephone line for transmission to said remote 
different from the described embodiments, without depar- user. 

ture from the spirit of the invention. For example, the RTCP 3. The method of claim 2, wherein said protocol com- 
protocol processing may be distributed among the gateway prises the Real-time Transfer Protocol, 
platform and the modem platform, for example the infor- 4. The method of claim 2, wherein said at least one 
mation for the RTCP send and receive reports comes from modem computing platform performs the processing of jitter 
the modem platform but the actual send and receive report buffering for said data communication, 
may be generated and sent out by either the modem or DSP 5. The method of claim 2, wherein said at least one 
platform or the gateway platform. As a further example, modem computing platform performs processing of frame- 
while the best mode known to the inventors for practicing 35 based audio data associated with said protocol, 
the invention has been disclosed in the context of present or <>• The method of claim 2, wherein said gateway comput- 
proposed commercial products of the applicants' assignee, it m S platform performs the processing of creation and termi- 
will be appreciated that the teachings are readily adaptable nation of audio streams associated with said protocol, 
to other types of network access servers marketed by others . T P? method of claim 2, wherein said gateway comput- 
in the industry, such as Livingston, Ascend, Cascade 20 P^oim performs the terminaUon of RTCP traffic. 

Communications, etc. This true spirit and scope of the f 8 * ™ e metI ?° d ^T,' wh ? r f m ™ th ° d 15 per ; 

* j c j i_ *i_ c ii • i . * i_ • * formed in an Internet telephony data session between said 

invention is defined by the following claims, to be inter- host ^ said nmoU . ^ 

preted in light of the above description. 0 ^ method M claimed ^ any Qne Qf claims ^ 

e claim. ,25 wherein said method is performed in a high density network 

1. A method for routing data between a host connected to ^ ^^sing a plurality of DSP cards connected 
a computer network and a remote user connected to said host b a h[ h ^ Uel bus tQ said nctwork mlcrfacc> cach 
via a te ephone lme, the data associated with a higher level of gaid Dsp cafds idin modem for all DS0 
protocol for aUowmg ^the hast to communicate with the channek of a T1 of m Unc 

remote user, said higher level protocol selected from the 3Q 1Q In a network acce&s for providing connectivity 

group of protocols consisting of the Point-to-Point Protocol between a ho&t sgmm of data tQ a network 

and the Serial Lme Interface Protocol, the method performed aQd & remote terminal connected to the blic switched 

in a communications access system comprising a telephone tel hone network , comprismg , m combination a telephone 

hne interface connected to said telephone line, a plurality of Uae tivel connecting said network access 

modems, each modem associated with a modem computing 35 semr tQ a telephone line carrying telephone signals; 

platform and a network interface having a gateway com- a of Dsp platforms for demo dulating 

putmg platform, compnsmg the steps of: inbound telephone signals and mo dulating outbound tele- 

receivmg said data destined for said remote user and phone signals for transmission on said telephone line to said 

directing said data from said network interface to one remote terminal; a bus passing digital telephone signals from 

of said modems in said communications access system; 4Q said telephone line interface to said DSP computing plat- 
distributing the processing of said higher level protocol f orms; and a network interface receiving data from said 

for said data, such that said gateway computing plat- modems via a bus and routing said data onto said network; 

form performs a first portion of the processing of said t he improvement comprising: 

higher level protocol and said modem computing plat- said nelwork imerface comprising a computing plat- 
form m said modem performs a second portion of the 45 form proccS sing a higher level protocol associated 
processing of said higher level protocol; and with comm unication between said remote terminal 
subsequently directing the data from said modem com- and sa j d host on said network, said higher level 
puting platform through said telephone line interface protocol selected from the group of protocols con- 
onto said telephone line for transmission to said remote sisting of the Point-to-Point Protocol and the Serial 
user - 50 Line Interface Protocol, and wherein each of said 

2. A method for routing data between a host connected to plurality of DSP computing platforms performs the 
a computer network and a remote user connected to said host processing of a subset of functions associated with 
via a telephone line, the data associated with a higher level sa j d higher level protocol associated with said corn- 
protocol for allowing the host to communicate with the munication. 

remote user, said higher level protocol comprising a real- 55 n \ n a network access server for providing connectivity 
time protocol associated with voice, video, and data between a host source of digital data connected to a network 
communication, the method performed in a communications m d a remote terminal connected to the public switched 
access system comprising a telephone line interface con- telephone network, comprising, in combination a telephone 
nected to said telephone line, a plurality of modems, each une interface operatively connecting said network access 
modem associated with a modem computing platform, and 60 scrver t0 a telephone line carrying digital telephone signals; 
a network interface having a gateway computing platform, a plurality of DSP computing platforms for demodulating 
comprising the steps of: inbound telephone signals and modulating outbound tele- 
receiving said data destined for said remote user and phone signals for transmission on said telephone line to said 
directing said data from said network interface to one remote terminal; a bus passing digital telephone signals from 
of said modems in said communications access system; 65 said telephone line interface to said DSP computing plat- 
distributing the processing of said higher level protocol forms; and a network interface receiving data from said 
for said data, such that said gateway computing plat- modems via a bus and routing said data onto said network; 
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the improvement comprising: 
said network interface comprising a computing plat- 
form processing a higher level protocol associated 
with communication between said remote terminal 
and said host on said network, wherein said higher 5 
level protocol comprises a Real-time Transport Pro- 
tocol (RTP), and wherein said plurality of DSP 
computing platforms performs the processing of a 
subset of functions associated with said higher level 
protocol. 10 

12. The improvement of claim 11, wherein said plurality 
of DSP computing platform performs the processing of RTP 
header generation. 

13. The improvement of claim 11, wherein said at least 
one modem computing platform performs the processing of 15 
a jitter buffer associated with said RTP. 

14. The improvement of claim 10, wherein said DSP 
computing platforms are organized into a plurality of high 
density modem cards, each modem card associated with all 
the DSO channels of a Tl or El span line. 20 

15. A method of performing the processing of the Real- 
time Transport Protocol (RTP) protocol in a communication 
session from a source connected to a computer network and 



a destination connected to a telephone line, the method 
performed in a network access server, comprising the steps 
of: 

passing RTP frames of data for said session from a 
gateway computing platform in said network access 
server having a network interface to a modem in said 
network access server; 

performing jitter buffering for said RTP frames of data in 
a computing platform in said modem; 

converting said RTP frames of data into a form suitable 
for transmission on said telephone line; and 

subsequently directing said data on said telephone line 
from said network access server to said destination. 

16. The method of claim 15, wherein said modem plat- 
form further processes all frame-based audio data associated 
with said session. 

17. The method of claim 15, wherein said method is 
performed in a network access server. 

18. The method of claim 15, wherein said network access 
server comprises a high density network access server. 
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